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Abstract. We show by construction that the procedure of local search for the problem 
of finding a local optimum of a quadratic pseudo-boolean function takes exponential time 
in the worst case, regardless of the neighbor selection and tie-breaking rules. 



1. Introduction 

We consider the problem of finding a local maximum of the quadratic pseudo-boolean 
function 

n n 

/(x) = f(xi, . . . ,x n ) =} j CjXj + ^ CijXiXj, x £ {0, l} n , Ci € R, Cjj € R. 

i=l l<i<j<n 

A straightforward approach is to use (increasing) local search, that is, start from an arbi- 
trary vertex of the boolean hypercube, and if it is not a local maximum, move to one of 
its feasible neighbors, i.e. to a neighbor where the function attains a larger value than at 
the current vertex. This method clearly works (terminates in a finite number of steps) for 
an arbitrary pseudo-boolean function, but its complexity is not clear in general. 

In this paper we show that the time complexity of the above local search algorithm is 
exponential regardless of the neighbor selection and tie-breaking rules. Note that we do not 
consider the problem of finding a good initial point, because with the smartest possible 
strategy we could choose a local optimum to be the initial point. Hence we assume that 
the local search always starts from an "arbitrary" vertex, which in this paper will be the 
origin. 

2. Previous Results 

In the survey paper [1] a few problems, including the Max-Cut problem, are listed 
which cannot be solved in (worst-case) sub exponential time by local search, but it does 
not contain any proofs. The two references given in [1] are [2] and [3], both of which 
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consider the Max-Cut problem. This is clearly a special case of maximizing a quadratic 
pseudo-boolean function, and hence such a result would be stronger than the one in this 
paper. Unfortunately, [2] is an unavailable manuscript. The paper [3] contains a proof of 
the claim that Max-Cut can be reduced with a tight PLS reduction to the Flip problem, 
which is known to be PLS-complete [1]. This in turn proves that local search is indeed 
exponential for Max-Cut. However, this proof is not constructive, as it does not allow us 
to construct a sequence of Max-Cut instances, increasing in size, for which local search 
takes exponentially increasing number of steps. The construction given in this paper, 
together with its proof, is considerably simpler, and it does not use the theory related to 
PLS-completeness. 

Another type of result is the one in [4]. It is proven that the fc-OPT Traveling 
Salesman problem is PLS-complete for some fixed k, and it is stated as a corollary (but 
without a separate proof) that there are instances of this problem for which the fc-opt 
local search takes exponential time. (Here k > 10000.) Let us note, however, that PLS- 
completeness does not imply the existence of exponentially hard instances. And even 
though the corollary is indeed true, the proof, again, does not lead to a construction. 

The aim of this paper is to provide a simple construction, which shows that for some 
instances of quadratic pseudo-boolean optimization every increasing local search algorithm 
takes exponential time to find a local maximum. 



3. The Construction 

Let us call a path (defined as a sequence of adjacent vertices) increasing if the objective 
function is monotone and nondecreasing along its vertices. In order to prove our claim 
it suffices to show that for a sequence of polynomials with increasing number of variables 
any increasing path that connects the origin to a local optimum has length exponential in 
the size of the polynomials. 

Let us define the sequence of n-variate polynomials {/„} (n = 2, 6, 10, ... ) recursively 
as follows. 

f2(xi,X 2 ) := X 1 + X 2 , 

/n+4(x, Xn+l ^Xn+2 j > x n +i) := /n(x) - M n n 2 x n+ i + M n {n + 1) x n+i~ 



- x n+2 - 2M n n ]^ Y] Xij x n+2 + 2M n n(n + 2)x n+ ix n+2 - 

~ 4 X ™+ 3 + 2x n+lXn+3 + 2x n+2 X n+3 - 3x ra+3 + 

+ (M n (n - 1) + 4) ( J^Xi ) x ra+ 4 + 6M n n 2 X n +3 x n+4 ~ 5M n n X n +4, 



where M n : = max f n — min f n + 1. Let p n denote the length of the shortest path local 
search may follow to a local maximum starting from the zero vector. (Eg. p2 = 2.) 
We can obtain the following bound for the size of the polynomials. 
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Lemma 1. The size of f n is polynomial in n. 

Proof. It is enough to show that the largest coefficient has size polynomial in n. Notice, 
that from the recursive definition, and because max/ n — min/ n can be bounded from 
above by the sum of the absolute values of the coefficients, M n+4 = 0(M n n 2 + poly(n)), 
consequently M n = O ((n!) 2 ), which shows that every coefficient of the polynomial can be 
represented by 0(n log n) bits. □ 

First we examine which local maximums may be accessible from the origin via an 
increasing path. 

Lemma 2. Every increasing path of f n starting from the origin ends at the antipodal of 
the origin, (1, 1, . . . , 1). 

Proof. The theorem holds if n = 2. We prove by induction that if it holds for f n , then it 
holds for / n +4 as well. 

One way to see this is to take the partial derivatives of /n+4 with respect to each variable, 
and check their signs at points which may be included in an increasing path. This solution 
can be summarized in a table, as shown in Table Q] on page Notice that for boolean 
functions the partial derivatives are the same as discrete partial derivatives, which give 
the change in the function value upon the change of one variable. Changing a variable 
from zero to one results in an increase if and only if the corresponding partial derivative is 
positive. Similarly, changing a variable from one to zero results in an increase if and only 
if the corresponding partial derivative is negative. Each but the last column of the table 
contains precisely one cell which indicates an increasing direction, implying that there is 
essentially one increasing path. In the first column it can be either the first or second cell, 
but for any fixed ^ only one of them. In the last column there is no partial derivative 
that indicates a possible increase. The increasing paths can hence be characterized based 
on this table: each of them first reaches the point (1, 1, . . . , 1, 0, 0, 0, 0), then reaches 
(1,1,..., 1, 1, 1, 0, 0) in two more steps, then in n steps goes to (0, 0, ... , 0, 1, 1, 0, 0), after 
which it goes to (0, 0, ... , 0, 1, 1, 1, 1), and finally it follows a path which is a translation 
of the first part of the path to arrive at (1, 1, . . . , 1, 1, 1, 1, 1), where the path ends. 

Alternatively, we can determine the increasing paths by directly going along the possible 
paths to verify that increasing paths end at the antipodal of the origin. To simplify this 
process, we hereby state the following properties of increasing paths of / n +4, which are all 
very easy to prove, and which we are going to use repeatedly. 

(1) // x n+ 2 = %n+3 = = 0, then we can change x n+ \ from to 1 with an 
increase in the objective function if and only if Xi = 1 for every i < n. This is 
because then the coefficient of x n +i is M n n > 0, while otherwise it is at most 
-M n (n 2 + (n - l)(n + 1)) < 0. 

(2) If x n +3 = x n +4 = 0, then we can change x n +2 from to 1 with an increase in the 
objective function if and only if x n+ \ = 1. This is trivial. 

(3) If x = 0, then we can change x n +3 from to 1 with an increase in the objective 
function if and only if x n+ \ = x n+ 2 = 1, and xi = for every i < n. The "if" 
direction is trivial, we can increase the objective function by one. To see that the 
"only if" part holds, observe that if x% = 1 for some i < n, then the objective 
function will increase by at most —4 + 2 + 2 — 3<0, and if x n +i or x n +2 is zero, 
then the increase is at most 2 — 3 < 0. 
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(4) We can change x n +4 from to 1 with an increase in the objective function if and 
only if x n+ 3 = 1. In the "if" direction observe that the coefficient of x ra+ 3X Tt+ 4 
is larger than the sum of the absolute values of every negative coefficient of the 
terms involving x n+ 4. On the other hand, if x n+3 = 0, then the term — 5M ra nx n+ 4 
is dominating. 

(5) An increasing path that enters the subcube defined by x ra+1 = x n+ 2 = 1 never leaves 
that subcube. Leaving the subcube would result in a decrease of the objective 
function, as the the coefficient of x n +ix n +2 dominates the sum of the absolute 
values of every negative coefficient of terms which contain these two variables. 

(6) An increasing path that enters the subcube defined by x ra+ 3 = x ra+ 4 = 1 never leaves 
that subcube. The proof is similar to the previous one. 

(7) If Xi = for every i < n and x n+ \ = x n +2 = 1, then we can change some Xi with 
i < n to 1 only if x ra+ 4 = 1. Because if x n+ 4 = 0, then changing Xj would result 
in an increase of less than M n (n + 2) due to the terms f n and XjX ra+ i, which is 
dominated by the decrease 2M n n due to the term XiX n +2- 

(8) For any x € {0, l} n , / n+ 4(x, 1, 1, 1, 1) = / n (x) + C n , where C n depends on n, but 
not on x. Substitute the values of the last four variables into the definition of f n+ 4 
and observe that every term in {xiXj \ i < n < j} cancels out. 

Consider now an increasing path of f n +A, starting at the origin. By the inductive 
assumption and the definition of f n +A as long as the path remains in the subcube defined 
by x n+ i = x ra+ 2 = x n+ 3 = x n+ 4 = 0, it coincides with an increasing path of /„. Claims 

1- 4 imply that it cannot leave the subcube until Xj = 1 for each i < n, and that when this 
condition is satisfied, there is exactly one possible direction the path can proceed, which 
is changing x n+ \ to 1. 

Hence, the path must arrive at vertex (1,1,0,0,0). Now we cannot change back x n+ \ 
to 0, because we never step back. Also, we cannot change any Xj with i < n back to 0, 
because the possible increase in the term /„ is less than M n , which does not compensate 
for the decrease in the term M n (n + l)xjX ri+ i. This observation, together with Claims 

2- 4, yields that the only feasible neighbor of the current vertex is (1, 1, 1, 0, 0). 

Claim 5 ensures that from now on the value of x n+ i and x ra+ 2 never change. Claims 3 
and 4 guarantee that the last two variables also cannot change until the first n variables 
are all set to zero. At the same time, due to the terms — 2M ra XjX n+ 2, any x« with i < n 
can be changed from 1 to with an increase of the objective function. Consequently in 
the next n steps the increasing path can only continue by changing every Xj, i < n, one 
by one, back to zero. (Clearly these steps cannot be interspersed by — ► 1 changes, as 
such steps would decrease the objective function, due again to the terms — 2M n XjX„+2-) 

Eventually we arrive at the vertex (0, 1, 1, 0, 0). According to Claim 5 the two compo- 
nents of value 1 will never change, and knowing this Claim 7 prevents the first n variables 
from changing their values until x n+ 4 is set to 1 . Now Claims 3 and 4 imply that the only 
feasible neighbor of the current vertex is (0,1,1,1,0), and the only feasible neighbor of 
that one is (0, 1, 1, 1, 1). 

Claims 5 and 6 now ensure that from now on only the first n variables may have their 
value changed, and Claim 8 implies that (x', 1, 1, 1, 1) is a feasible neighbor of (x, 1, 1, 1, 1) if 
and only if (x', 0, 0, 0, 0) is a feasible neighbor of (x, 0, 0, 0, 0). Consequently the remaining 
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subpath of the path being considered is a translation of an increasing path of f n . Using 
our inductive assumption the increasing path of / n +4 also ends at (1,1,. ..,1). □ 

Reviewing the previous proof one can observe that we have also proved our main theo- 
rem. 

Theorem 1. Any local search algorithm starting from the origin takes exponentially many 
steps (both in n and in the size of f n ) to find a local maximum of the polynomial f n . 
Proof. It is enough to show that the shortest increasing path, corresponding to /„, con- 
necting the origin to its antipodal has length exponential in n. Then from Lemma 1 it 
also follows that the path length is exponential in the size of /„, too. 

In the previous proof we have seen that every increasing path corresponding to / n +4 
contains two disjoint subpaths which are isomorphic to some increasing path corresponding 
to /„. Consequently p n +4 > 2p n , which yields (since p2 = 2) the inequality p n > 2 ra / 4 . □ 

Even simpler constructions can be given for special cases of the problem (in terms of the 
heuristics used). An interesting special case is the when the greedy local search procedure 
is applied: in this case the neighbor selection rule is to choose the one with the highest 
objective function value. It is left to the reader to apply any of the techniques used in the 
proof of Theorem 1 to prove the following. 

Theorem 2. The greedy local search algorithm starting from the origin takes exponentially 
many steps (both in n and in the size of g n ) to find a local maximum of the polynomial g n 
(n G {2, 6, 10, ... }; defined by: 

92(xi,x 2 ) :=2x 1 +x 2 , 

n 

Sf n+4 (x, x n+1 ,x n+2 ,x n+3 , x n+A ) :=4# n (x) + 3x n+ i - ^(2M n + i + 2)xiX n+2 

i=i 

+ 3M n nx n+1 x n+2 + 2x n+3 + M n x n+1 x n+3 
- 4M n nx n+2 x n+ 4 + 5M n nx n+3 x n+4: , 

where M n := 4 (maxg n — ming n ). 

Starting from the zero vector, the path the greedy local search procedure might follow 
is unique (i.e., there are no ties), the path ends at the unique local maximum (which is 
the complement of the characteristic vector of the set {4, 8, • • • , n — 2}), and it has length 
greater than 2 n / 4 . 
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Table 1. Signs of partial derivatives at specific points. 5 := J27=i x i 



